Image processing method and image processing apparatus

ABSTRACT

According to one embodiment, an image processing method causes a computer to function as an image processing apparatus including an acquisition unit and a control unit. The acquisition unit acquires image data. The control unit performs a control process of detecting a character string of each color included in the image data and associating the character string with the color using the character string of the color.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2019-056523, filed Mar. 25, 2019, theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image processingmethod and an image processing apparatus.

BACKGROUND

Some image processing systems perform an optical character recognition(OCR) process on the entire region or a designated region of image datato acquire a character string. The character string acquired through theOCR process may be further subjected to an operation such as copying orstoring. However, after the OCR process, it is necessary to perform anoperation on the acquired character string. This operation addscomplexity to image processing and is undesirable.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a hardware configuration exampleof an image processing apparatus according to an embodiment;

FIG. 2 is a schematic block diagram illustrating a functionalconfiguration example of the image processing apparatus according to afirst embodiment;

FIG. 3 is a diagram illustrating an example of a document image;

FIG. 4 is a diagram illustrating a first example of parameters;

FIG. 5 is a diagram illustrating a process flow of the image processingapparatus;

FIG. 6 is a diagram illustrating a second example of parameters;

FIG. 7 is a diagram illustrating an example of data output by the imageprocessing apparatus; and

FIG. 8 is a diagram illustrating a process flow of an image processingsystem according to a second embodiment.

DETAILED DESCRIPTION

Embodiments generally relate to an image processing method and an imageprocessing apparatus which reduce the complexity of an operation on acharacter string acquired through an OCR process.

In general, according to one embodiment, an image processing methodcauses a computer to function as an image processing apparatus includingan acquisition unit and a control unit. The acquisition unit acquiresimage data. The control unit performs a control process of detecting acharacter string of each color included in the image data andassociating the character string with the color using the characterstring of the color.

Hereinafter, an image processing method and an image processingapparatus according to an embodiment will be described with reference tothe drawings.

FIG. 1 is a block diagram illustrating an apparatus configurationexample of an image processing system including an image processingapparatus 100 according to the embodiment. The image processing systemillustrated in FIG. 1 includes the image processing apparatus 100, animage forming apparatus 200, and an information processing apparatus300. The image processing apparatus 100 is connected to the imageforming apparatus 200 via a communication cable.

The image forming apparatus 200 is, for example, a multi-functionalperipheral. The image forming apparatus 200 includes an image readingunit 201. The image reading unit 201 optically reads a target documentimage to generate image data with color information. The image readingunit 201 is configured using an image sensor, such as a charge coupleddevice (CCD) or complementary metal oxide semiconductor (CMOS) sensor.The image reading unit 201 outputs the generated image data to the imageprocessing apparatus 100.

The image processing apparatus 100 acquires the image data output by theimage reading unit 201. The image processing apparatus 100 performs acontrol process of detecting a character string of each color includedin the image data and associating the character string for each colorusing the character string of the color. The image data includes colorinformation. The image data also includes an image of the characterstring. In other words, the image data is, for example, image data of adocument image in which the character string is described. The imagedata may be image data of a screen of a whiteboard. The image data maybe image data generated through imaging in an imaging apparatus.

The image processing apparatus 100 is a computer that includes a centralprocessing unit (CPU) 101, a communication module 102, and an interface103. The image processing apparatus 100 further includes a random accessmemory (RAM) 104, a read-only memory (ROM) 105, a hard disk drive (HDD)106, and the like. The communication module 102 may perform wirelesscommunication or wired communication with each of the image formingapparatus 200 and the information processing apparatus 300. The RAM 104stores an image processing program 107 and a parameter table 108.

First Embodiment

FIG. 2 is a schematic block diagram illustrating a functionalconfiguration example of the image processing apparatus 100 according toa first embodiment.

The CPU 101 (see FIG. 1) reads the image processing program 107 (seeFIG. 1) stored in the ROM 104 or the like and executes the imageprocessing program. Thus, the image processing apparatus 100 realizeseach function of an acquisition unit 110, a region determination unit111, a character recognition unit 112, and a control processing unit113. Some of the functions 110 to 113 may be realized by an electroniccircuit in the image processing apparatus 100.

The acquisition unit 110 acquires image data of a document image. Theregion determination unit 111 detects a color region of each color fromthe image data. The color region indicates a range in which a characterstring is described with a similar color to that of the image data. Inother words, the color region indicates a region including a unity ofthe character string with the similar color in the image data. The colorregion may be a range in which the character string is decorated withthe similar color in the image data. For example, the decoration isrealized by drawing a line to overlap the character string orsurrounding the character string with a highlighter, a marker, or thelike. In the embodiment, the similar color includes a range of the colorconsidered to be a given color. In addition, the similar color includesa range of the color specified with the name of a color such as “red” or“blue.”

The character recognition unit 112 detects a character string (textdata) included in each color region through an OCR process with regardto the each color region detected by the region determination unit 111.Specifically, the character recognition unit 112 recognizes charactersin each color region and specifies which character each character is.The character recognition unit 112 detects a character string includinga specific character. The character string may include only onecharacter. The character may include a numerical value, a mathematicalformula, a sign, or the like.

The control processing unit 113 acquires a control process associatedwith each color with reference to the parameter table 108. The controlprocessing unit 113 performs the control process associated with eachcolor using the character string recognized in the color region of thecolor.

An example of the parameter table 108 will be described below withreference to FIG. 4.

FIG. 3 is a diagram illustrating an example of a document image 10according to the embodiment. The image processing apparatus 100 performsa process using image data of the document image 10 illustrated in FIG.3 as an input. The document image 10 illustrated in FIG. 3 includescharacter strings and diagrams, such as graphs. The character stringsincluded in the document image 10 include, for example, a document name,a date, a division name, and sentences.

Specifically, in the document image 10 illustrated in FIG. 3, each unityof the character string is described with a different color. Forexample, in a color region 11 a of the document image 10, a documentname “ABC document” is described with red. In a color region 11 b, adate “YYYY/MM/DD” is described with black and a line is drawn on thedate with a yellow highlighting marker. In a color region 11 c, adivision name “DDD business division” is described with blue. In colorregions 11 d and 11 f, each document is described with black. In colorregions 11 e and 11 g, each document is described with black and thedocument is surrounded by a green marker. For example, in the colorregions 11 e and 11 g, documents are surrounded by ellipses, rectangles,or the like.

In this way, the document image 10 in FIG. 3 includes images of thecharacter strings with a plurality of colors. The document image 10illustrated in FIG. 3 includes the character strings with five colors.The number of colors of the character strings included in the documentimage 10 is not limited to this example. The document image 10 in theembodiment may include images of character strings with two or morecolors.

FIG. 4 is a diagram illustrating an example of the parameter table 108according to the embodiment. The parameter table 108 has a correspondentrelation between colors and control processes.

In the parameter table 108 illustrated in FIG. 4, a control process“STORED AS CONTENT OF FILE 1” is associated with color “black.” Acontrol process “DESIGNATED AS FILE NAME OF FILE 1” is associated withcolor “red.” A control process “DESIGNATED AS FOLDER NAME” is associatedwith color “blue.” A control process “STORED AS CONTENT OF FILE 2” isassociated with color “green.” A control process “DESIGNATED AS FILENAME OF FILE 2” is associated with color “yellow.”

That is, in the parameter table 108 illustrated in FIG. 4, File 1 thatstores a character string of a color region of black is designated sothat the character string of the color region of red is generated as afile name. Further, File 2 that stores a character string of the colorregion of green is designated so that the character string of the colorregion of yellow is generated as a file name. Files 1 and 2 aredesignated so as to be generated under a folder with a folder name ofthe character string of the color region of blue.

In the embodiment, the parameter table 108 is stored in advance in theimage processing apparatus 100 as will be exemplified, but an exemplaryembodiment is not limited to this example. The parameter table 108 maybe input to the image processing apparatus 100 along with the imagedata.

The parameter table 108 may be different for each piece of image data ormay be used in common between a plurality of pieces of image data.Information of the parameter table 108 may be input by a user in thecontrol process.

In the parameter table 108 illustrated in FIG. 3, content of the controlprocess is associated with each color, but an exemplary embodiment isnot limited to this example. In the parameter table 108, the content ofthe control processes may be associated with only some of the pluralityof colors.

FIG. 5 is a diagram illustrating a process flow of the image processingapparatus 100 according to the embodiment. The acquisition unit 110 ofthe image processing apparatus 100 acquires image data (ACT11).Specifically, the acquisition unit 110 acquires the image data output bythe image reading unit 201 via the communication module 102. Theacquisition unit 110 may acquire the image data stored in a storagedevice such as the HDD 106.

The region determination unit 111 extracts a color region of each colorin the image data acquired by the acquisition unit 110 (ACT12). Theregion determination unit 111 extracts a range in which a characterstring is described with the same color in the image data as a colorregion. The region determination unit 111 extracts a range in which acharacter string is decorated with the same color in the image data as acolor region. For each color, the region determination unit 111 detectsthe color region in which the character string is described with thecolor from the image data.

In the example of the document image in FIG. 3, the region determinationunit 111 respectively detects the color regions 11 a to 11 g from theimage data of the document image 10. Hereinafter, when not beingdistinguished, the color regions 11 a to 11 g are referred to as thecolor regions 11. Specifically, the region determination unit 111extracts each color region 11 as follows, for example. The regiondetermination unit 111 divides the image data into tiles with apredetermined size and acquires an average value of the colorinformation of each tile. The tile includes a plurality of adjacentpixels. The color information is, for example, red-green-blue (RGB)values. The region determination unit 111 compares the average value foreach tile to the average value of adjacent tiles to determinedifferences (e.g., difference values, etc.) between the average value ofthe tile and the average value of each adjacent tile. The regiondetermination unit 111 then compares the differences to a predeterminedrange. When a difference is within the predetermined range, the adjacenttile associated with the difference is termed a “detected tile.”. Theregion determination unit 111 detects a region including the pluralityof detected tiles as the color region 11. The region determination unit111 detects the color of the detected color region 11 based on the imagedata.

The region determination unit 111 may detect each color region 11 inaccordance with another method. Similarly, the region determination unit111 divides the image data into tiles with a predetermined size andacquires an average value of the color information of each tile. Theregion determination unit 111 maintains a sample value of the colorinformation of each color (for example, black, red, blue, green, andyellow) in advance. The region determination unit 111 detects aplurality of adjacent tiles in which a difference between the averagevalue and a sample value of the color information of each color iswithin a predetermined range. The region determination unit 111 detectsa region including the plurality of detected tiles as the color region11.

The region determination unit 111 may detect the color region 11surrounded by a marker or the like as follows. The region determinationunit 111 acquires the color information of each pixel while scanningeach pixel of the image data in sequence from the top left end. Theregion determination unit 111 determines that pixels of similar colorinformation continue when the difference in the color information of thepixels of the before and after in a scanning sequence is within thepredetermined range. In this case, the region determination unit 111determines whether the pixels in the periphery of the pixels of thesimilar color information that continue have the similar colorinformation. The region determination unit 111 detects a pixel groupdemarcating a rectangle, an ellipse, or the like with the similar colorinformation based on the positions of the peripheral pixels with thesimilar color information. The region determination unit 111 detects thecolor region 11 based on the detected pixel group.

A method of detecting the color region 11 is not limited to theabove-described example. The region determination unit 111 can detecteach color region 11 in the image data in accordance with any of variousgenerally known technologies of image processing.

The character recognition unit 112 performs the OCR process on eachcolor region 11 detected by the region determination unit 111. Thus, thecharacter recognition unit 112 detects the character string included ineach color region 11 as text data (ACT13). In the example of thedocument image in FIG. 3, the character recognition unit 112 detects thecharacter strings included in the color regions 11 a to 11 g of theimage data. Specifically, the character recognition unit 112 performsthe OCR process of the color region 11 a to detect the character string“ABC document.” The character recognition unit 112 similarly detectscharacter strings in other color regions. Thus, the character string ofeach color in the image data is detected. As described above, thecharacter string of each color includes a character string describedwith a predetermined color and a character string decorated with apredetermined color.

In the process flow of FIG. 5, the character recognition process (ACT13)performed after the process of detecting the color regions 11 (ACT12)will be exemplified. However, an exemplary embodiment is not limited tothis example. For example, by performing the process of detecting thecolor region 11 (ACT12) after the character recognition process (ACT13),the character string of each color may be detected from the image data.

In this case, for example, the character recognition unit 112 performsthe OCR process on the entire image data to detect the characterstrings. The region determination unit 111 acquires each characterstring and the color information of the peripheral pixels of thecharacter string based on the image data. The region determination unit111 detects the character string of the similar color information amongthe continuous character strings. Alternatively, the regiondetermination unit 111 detects the character string surrounded with thesimilar color information among the continuous character strings. Thus,the character string of each color included in the image data can bedetected.

The control processing unit 113 performs the control process associatedwith each color using the character string of the color with referenceto the parameter table 108 (ACT14). Specifically, the control processingunit 113 acquires content of the control process associated with eachcolor with reference to the parameter table 108. The control processingunit 113 acquires the character string of each color detected by thecharacter recognition unit 112. The control processing unit 113 performsthe control process associated with the color of the acquired characterstring using the character string.

Here, an example of a process (ACT14) of the control processing unit 113will be described with reference to the document image 10 in FIG. 3 andthe parameter table 108 in FIG. 4.

As described above, in the document image 10 in FIG. 3, the documentname “ABC document” is described with red in the color region 11 a. Inthe color region 11 c, the division name “DDD business division” isdescribed with blue. In color regions 11 d and 11 f, the documents aredescribed with black. The control processing unit 113 acquires contentof the control process associated with colors “red,” “blue,” and “black”with reference to the parameter table 108. As described above, thecontrol process “STORED AS CONTENT OF FILE 1” is associated with color“black.” The control process “DESIGNATED AS FILE NAME OF FILE 1” isassociated with color “red.” The control process “DESIGNATED AS FOLDERNAME” is associated with color “blue.”

Accordingly, the control processing unit 113 generates a file thatstores (e.g., is populated with, etc.) the documents of the colorregions 11 d and 11 f. The control processing unit 113 sets the filename of the generated file in the character string “ABC document.” Thecontrol processing unit 113 stores the generated file under the folderof the folder name “DDD business division.” The folder “DDD businessdivision” may be an existing folder or a newly generated folder.

In the embodiment, the name of the folder (the DDD business division)designated as the folder name is exemplified, as described above, but anexemplary embodiment is not limited to this example. A full path nameindicating the name and the hierarchy of the folder may be designated asthe folder name.

Similarly, in the color region 11 c, a line is drawn on the date“YYYY/MM/DD” with the yellow highlighting marker and the documents ofthe color regions 11 e and 11 g are surrounded with a green marker. Thecontrol processing unit 113 generates the file that stores the characterstrings of the color regions 11 e and 11 g and sets the file name of thegenerated file in the character string “YYYY/MM/DD.” The controlprocessing unit 113 stores the generated file under the folder “DDDbusiness division.”

In this way, in the embodiment, the control process of using thecharacter string as the file name is associated with the first color andthe control process of storing the character string in the file isassociated with the second color. In the image processing apparatus 100,the user who generates the file that stores the character string of thesecond color and has the character string of the first color as the filename generates, for example, the document image 10 in which a desiredcharacter string is described or decorated with a predetermined color asthe file name or the folder name.

For example, the user may write the character string with thepredetermined color on the printed document image 10. The user maydecorate the character string of the printed document image 10 with thepredetermined color. Thus, the image processing apparatus 100 cangenerate the file that has the desired name and stores the characterstring included in the image data.

The document image 10 may include a handwritten character string anddecoration. For example, when a discussion is performed based on thedocument image 10, information is written with a red or blue pen on thedocument image 10 in some cases. For example, the user writes thecharacter string desired to be designated as the file name with a redpen or the like on the document image 10.

The user surrounds the character string within the document image 10desired to be stored as another file with a green marker or the like.Thus, the image processing apparatus 100 can generate a file that has afile name designated with red and has a document described in advance inthe document image 10 as content. The image processing apparatus 100 canstore a character string described with green in the document image 10in another file.

As described above, the image data may be image data of a printed imageof a whiteboard. Character strings are written with markers of aplurality of colors on the whiteboard. For example, importantinformation is written with a red marker. In this case, for example, theimage processing apparatus 100 can efficiently utilize the importantinformation by associating a predetermined control process with the redcharacter string.

Modification Example 1

Another example of the control process will be described in ModificationExample 1.

Modification Example 1 may be used in combination of the firstembodiment.

FIG. 6 is a diagram illustrating an example of a parameter table 108-1according to Modification Example 1. As described in FIG. 4, theparameter table 108-1 has a correspondent relation between colors andcontent of control processes. In the parameter table 108-1 illustratedin FIG. 6, a process of associating predetermined identificationinformation with a character string corresponds for each color.Specifically, association with identification information “ID01”corresponds to color “black.” Association with identificationinformation “ID02” corresponds to color “red” and association withidentification information “ID03” corresponds to color “blue.”

The control processing unit 113 outputs the identification informationin accordance with the color in association with a character stringacquired through the OCR process. Accordingly, the control processingunit 113 associates the identification information “ID01” with thedocuments of the color regions 11 d and 11 f and associates theidentification information “ID02” with the character string “ABCdocument.” The control processing unit 113 associates the identificationinformation “ID03” with the character string “DDD business division.”

In Modification Example 1, the control processing unit 113 outputs eachcharacter string associated with the identification information to thesame file, as an example.

FIG. 7 is a diagram illustrating an example of an output file accordingto Modification Example 1. In an output file 20 illustrated in FIG. 7,the identification information “ID01” is given in the beginning of thedocuments of the color regions 11 d and 11 f. Similarly, theidentification information “ID02” is given in the beginning of thecharacter string “ABC document” and the identification information“ID03” is given in the beginning of the character string “DDD businessdivision.”

In this way, the identification information is given to the characterstring. Thus, for example, the user can identify a type of eachcharacter string based on the identification information with referenceto the output file 20. Another application can read the output file 20and register the character strings in a database based on theidentification information. For example, the other application registerseach character string of the output file 20 as data in items inaccordance with the identification information of the database.

The identification information is not limited to the example illustratedin FIG. 6. For example, the identification information may be a tag of amarkup language. In this case, for example, information given in thebeginning of the character string and identification informationincluding information given to the end of the character string aredesignated. Thus, the control processing unit 113 can generate a file inwhich a predetermined tag is given to each character string after theOCR process.

In this way, in Modification Example 1, a control process of addingpredetermined identification information to a character string to outputthe predetermined identification information is associated with thirdcolor. The image processing apparatus 100 detects the character stringwith the third color in the image data and adds (e.g., appends, etc.)predetermined identification information to the character string tooutput the predetermined identification information. That is, the imageprocessing apparatus 100 adds the predetermined identificationinformation to a character string with a color associated with thepredetermined identification information to output the predeterminedidentification information. For example, the user generates the documentimage 10 in which the character string associated with desiredidentification information is described or decorated in accordance withthe color corresponding to the identification information. Thus, theimage processing apparatus 100 can give the predetermined identificationinformation in accordance with the color to the character string basedon the image data of the document image 10.

Modification Example 2

Another example of the control process will be described in ModificationExample 2.

Modification Example 2 may be used in combination of one or both of thefirst embodiment and Modification Example 1.

Although not illustrated, in Modification Example 2, output of acharacter string to a predetermined output destination is associated foreach color as content of the control process in the parameter table 108.The output destination is, for example, another application operating onthe image processing apparatus 100. The output destination may be aprocess, a thread, or the like operating on the image processingapparatus 100. The output destination may be an existing file.

In the parameter table 108, for example, output of an application to aprocess A is associated with color “black.” Output of a process B of anapplication is associated with color “red.”

Accordingly, the control processing unit 113 outputs the documents ofthe color regions 11 d and 11 f to the process A of the application. Thecontrol processing unit 113 outputs the character string “ABC document”to the process B of the application. For example, the control processingunit 113 outputs the character string to a process of an outputdestination through communication between the processes.

In this way, in Modification Example 2, the control process ofoutputting the character string to the predetermined output destinationis associated with the third color. The image processing apparatus 100detects the character string with the third color in the image data andoutputs the character string to the predetermined output destination.That is, the image processing apparatus 100 outputs the character stringof the color associated with the output to the predetermined outputdestination to the predetermined output destination. For example, theuser generates the document image 10 in which a character string desiredto be output to a desired output destination is described or decoratedin accordance with the color corresponding to the output destination.Thus, the image processing apparatus 100 can output the character stringto a corresponding output destination, based on the image data of thedocument image 10.

In this way, the image processing apparatus 100 according to the firstembodiment includes the acquisition unit 110 that acquires the imagedata and the control units (111 to 113). The control units detects thecharacter string of each color included in the image data and performsthe control process associated with the color using the character stringof the color.

Thus, the control process for the character string acquired through theOCR process in the image data of the document image 10 can be designatedin advance. In other words, the control process for the character stringacquired through the OCR process can be designated in accordance with acolor. The image processing apparatus 100 can perform the controlprocess associated with the color using the character string of eachcolor based on the color of the character string included in the imagedata. Accordingly, the user can perform the process in accordance withthe color without performing an operation after the OCR process byincluding the character string of a predetermined color in the documentimage 10. Thus, it is possible to reduce complexity of an operation onthe character string acquired through the OCR process.

The image processing apparatus 100 can cause the control process afterthe OCR process to be different for each color corresponding to thecharacter string. Thus, any control process can be designated for eachcolor. By generating the parameter table 108 for each piece of imagedata, it is possible to cause the control process to be different inaccordance with the image data.

In the embodiment, the image processing apparatus 100 is separate fromthe image forming apparatus 200, as exemplified above, but the imageprocessing apparatus 100 may be included in the image forming apparatus200 in other embodiments. In this case, after a process of reading thedocument image 10 is performed by the image reading unit 201, a processof the image processing apparatus 100 according to the embodiment isperformed. That is, after the document image is read, the controlprocess corresponding to the color of a character string can beperformed using the character string included in the image data. Thus,it is possible to reduce complexity of an operation on the characterstring of the image data.

Second Embodiment

The image processing apparatus 100 according to a second embodimentperforms a control process in cooperation with another apparatus. Aconfiguration example of an image processing system according to thesecond embodiment is similar to that of the first embodiment (see FIG.1). A functional configuration of the image processing apparatus 100according to the second embodiment is also similar to that of the firstembodiment (see FIG. 2). Process content of the control processing unit113 according to the second embodiment is different from that of thefirst embodiment. Hereinafter, process content of the control processingunit 113 according to the second embodiment will be described withreference to a process flow.

FIG. 8 is a diagram illustrating a process flow of an image processingsystem according to the embodiment. The information processing apparatus300 according to the embodiment activates a cooperation program (notillustrated) that performs a process in cooperation with the imageprocessing program 107. The cooperation program is, for example, aprogram that collects and manages data and classifies or analyzes thedata. The cooperation program stores, for example, various kinds of datain a database (not illustrated).

The user logs in the image forming apparatus 200 (ACT21). Specifically,the image forming apparatus 200 displays a screen for receiving an inputof an identification (ID) and a password. When the user inputs the IDand the password, the image forming apparatus 200 performs anauthentication process. When the authentication succeeds, the imageforming apparatus 200 performs an instruction to activate the imageprocessing program 107 of the image processing apparatus 100.

The image processing program 107 of the image processing apparatus 100performs an activation process (ACT22). The image forming apparatus 200notifies the activated image processing program 107 of the input ID andpassword. The image processing program 107 notifies the informationprocessing apparatus 300 of the ID and password notified by the imageforming apparatus 200. The cooperation program operating in theinformation processing apparatus 300 performs an authentication processbased on the ID and the password notified by the image processingapparatus 100 (ACT23). When the authentication succeeds, the cooperationprogram notifies the image forming apparatus 200 of informationindicating success of the authentication.

When receiving the notification, the image forming apparatus 200instructs the image reading unit 201 to read the document image 10. Theimage reading unit 201 reads the document image 10 and generates imagedata (ACT24). The image reading unit 201 transmits the generated imagedata to the image processing apparatus 100.

The image processing apparatus 100 acquires the image data transmittedfrom the image forming apparatus 200 (ACT25). The processes of ACT25 andACT26 are similar to the processes of ACT12 and ACT13 described in FIG.5. The control processing unit 113 performs the control processassociated with the color using the character string of the color withreference to the parameter table 108 (ACT28).

In the embodiment, an application programming interface (API) called ascontent of the control process corresponds to the parameter table 108for each color. The API is an API of the cooperation program. The imageprocessing program 107 can use each function of the cooperation program,data to be managed, or the like by calling the API of the cooperationprogram.

The API of the cooperation program is, for example, a command toregister data, update data, or acquire data. Specifically, for example,a first API is a command to input data to an item X managed by thecooperation program. A second API is a command to input data to an itemY managed by the cooperation program. In the parameter table 108, forexample, calling of the first API is associated with color “black.”Calling of the second API is associated with color “green.”

When the control processing unit 113 calls the API associated with thecolor of the character string using the character string, thecooperation program performs a process of inputting the character string(ACT29). In the example of the document image 10 described in FIG. 3,the control processing unit 113 calls the first API using the characterstrings of the color regions 11 d and 11 f.

The control processing unit 113 calls the second API using the characterstrings of the color regions 11 e and 11 g. Thus, the character stringsof the color regions 11 d and 11 f are input to the item X of the dataof the cooperation program and the character strings of the colorregions 11 e and 11 g are input to the item Y.

In this way, for example, the user generates the document image 10 inwhich a character string desired to be input to each item of thecooperation program is described with a color corresponding to the item.Thus, the image processing apparatus 100 can input a character stringafter the OCR process as data of each item managed by the cooperationprogram.

As described above, when a discussion is performed using the documentimage 10, the user writes information with a red or blue pen on thedocument image 10 in some cases. The image processing apparatus 100acquires the image data of the document image 10 written by the user.

Thus, the image processing apparatus 100 can efficiently input thecharacter string written with red or blue by the user to a predetermineditem of the data of the cooperation program.

The API is not limited to the above-described content. The API may be,for example, a command to transmit an electronic mail. The API may be acommand to output a character string to a predetermined process orthread of the cooperation program. In this case, the control processingunit 113 can output the character string described with a predeterminedcolor to the process of the cooperation program associated with thecolor.

As described above, in the second embodiment, a control process ofcalling a predetermined API using a character string is associated witha fourth color. The image processing apparatus 100 includes theacquisition unit 110 that acquires the image data and the control units(111 to 113).

The control unit detects the character string of each color included inthe image data and calls a predetermined API using the character stringof the fourth color. That is, the image processing apparatus 100performs the API associated with a predetermined color using thecharacter string with the predetermined color.

Thus, the image processing apparatus 100 can perform the API associatedwith the color using the character string of each color based on thecolor of the character string included in the image data.

The user generates the document image 10 in which the character stringused in calling of the desired API is described or decorated with thecolor corresponding to the desired API. Thus, the image processingapparatus 100 can generate a desired API using a desired characterstring without performing an operation after the OCR process based onthe image data of the document image 10.

Thus, it is possible to reduce complexity of an operation on thecharacter string acquired through the OCR process.

The second embodiment may be used in combination of the first embodimentand Modification Examples 1 and 2. By using the embodiments incombination, it is possible to realize the control process using eachcharacter string of the image data more appropriately.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the present disclosure. Indeed, the novel embodiments describedherein may be embodied in a variety of other forms: furthermore variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of thepresent disclosure. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the present disclosure.

1-14. (canceled)
 15. An image processing method implemented by an imageprocessing apparatus including an acquirer that acquires image data anda controller, the method comprising: acquiring, by the acquirer, imagedata; determining, by the controller, that a first color is included inthe image data; detecting, by the controller, a first character stringin the image data, the first character string being of the first color;determining, by the controller, a first application programminginterface based upon the first character string; and calling, by thecontroller, the first application programming interface.
 16. The methodof claim 15, further comprising: creating, by the controller, a filehaving a file name; determining, by the controller, that a second coloris included in the image data; detecting, by the controller, a secondcharacter string in the image data, the second character string being ofthe second color; and setting, by the controller, the file name as thesecond character string.
 17. The method of claim 16, further comprising:determining, by the controller, that a third color is included in theimage data; detecting, by the controller, a third character string inthe image data, the third character string being of the third color;determining, by the controller, identification information based uponthe third character string; appending, by the controller, the thirdcharacter string with the identification information to form an appendedcharacter string; and outputting, by the controller, the appendedcharacter string.
 18. The method of claim 16, further comprising:determining, by the controller, that a third color is included in theimage data; detecting, by the controller, a third character string inthe image data, the third character string being of the third color;determining, by the controller, an output destination based upon thethird character string; and outputting, by the controller, the thirdcharacter string to the output destination.
 19. The method of claim 18,further comprising: determining, by the controller, that a fourth coloris included in the image data; detecting, by the controller, a fourthcharacter string in the image data, the fourth character string being ofthe fourth color; determining, by the controller, a second applicationprogramming interface based upon the fourth character string; andcalling, by the controller, the second application programminginterface.
 20. The method of claim 16, further comprising populating, bythe controller, the file with the first character string.
 21. The methodof claim 15, wherein the first application programming interfacefacilitates input of data associated with a first managed item.
 22. Themethod of claim 21, further comprising: determining, by the controller,that a second color is included in the image data; detecting, by thecontroller, a second character string in the image data, the secondcharacter string being of the second color; determining, by thecontroller, a second application programming interface based upon thesecond character string; and calling, by the controller, the secondapplication programming interface.
 23. The method of claim 22, whereinthe second application programming interface facilitates input of dataassociated with a second managed item.
 24. An image processing apparatuscomprising: an acquirer configured to acquire image data; and acontroller configured to: determine that a first color is included inthe image data, detect a first character string in the image data basedupon the first color, determine a first application programminginterface based upon the first character string, and call the firstapplication programming interface.
 25. The image processing apparatus ofclaim 24, wherein the controller is further configured to: create a filehaving a file name; determine that a second color is included in theimage data; detect a second character string in the image data basedupon the second color; and set the file name as the second characterstring.
 26. The image processing apparatus of claim 25, wherein thecontroller is further configured to: determine that a third color isincluded in the image data; detect a third character string in the imagedata based on the third color; determine identification informationbased upon the third character string; and append the third characterstring with the identification information to form an appended characterstring.
 27. The image processing apparatus of claim 25, wherein thecontroller is further configured to: determine that a third color isincluded in the image data; detect a third character string in the imagedata based on the third color; determine an output destination basedupon the third character string; and output the third character stringto the output destination.
 28. The image processing apparatus of claim27, wherein the controller is further configured to: determine that afourth color is included in the image data; detect a fourth characterstring in the image data based on the fourth color; determine a secondapplication programming interface based upon the fourth characterstring; and call the second application programming interface.
 29. Theimage processing apparatus of claim 25, wherein the controller isfurther configured to populate the file with the first character string.30. The image processing apparatus of claim 24, wherein: the firstapplication programming interface facilitates input of data associatedwith a first managed item; and the controller is further configured to:determine that a second color is included in the image data, detect asecond character string in the image data, the second character stringbeing of the second color, determine a second application programminginterface based upon the second character string, and call the secondapplication programming interface.
 31. An apparatus comprising: anacquirer configured to acquire image data; and a controller configuredto: determine that a first color is included in the image data,determine a first character string in the image data that is of thefirst color, create a first file having a first file name, populate thefirst file with the first character string, determine that a secondcolor is included in the image data, determine a second character stringin the image data that is of the second color, determine a applicationprogramming interface based upon the second character string, and callthe application programming interface.
 32. The apparatus of claim 31,wherein the controller is further configured to: determine that a thirdcolor is included in the image data; determine a third character stringin the image data that is of the third color; determine if the thirdcolor is associated with the first color or the second color; and setthe third character string as the first file name in response todetermining that the third color is associated with the first color. 33.The apparatus of claim 32, wherein the controller is further configuredto: determine that a fourth color is included in the image data;determine a fourth character string in the image data that is of thefourth color; determine if the fourth color is associated with the firstcolor or the second color; and set the fourth character string as thefirst file name in response to determining that the fourth color isassociated with the first color.
 34. The apparatus of claim 32, whereinthe controller is further configured to: create a folder having a foldername; locate the first file in the folder; determine that a fourth coloris included in the image data; determine a fourth character string inthe image data that is of the fourth color; determine if the fourthcolor is associated with the third color; and set the fourth characterstring as the folder name in response to determining that the fourthcolor is associated with the third color.